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Overview 


Most students in the United States who enter community college with limited basic math skills face 
substantial challenges in completing their math requirements. Many of those students — who, by and 
large, must begin their studies with developmental (remedial) courses — drop out before ever becom- 
ing qualified to take college-level math courses. 


In 2014 the Texas Higher Education Coordinating Board (THECB) implemented major new guide- 
lines in how students who score very low in their course placement exams should be placed and served 
in community college courses. After adopting a new placement test, the state raised the cut score for 
requiring students to enroll in developmental education and eliminated the lowest-level developmental 
math course. Colleges were then permitted to choose among several alternatives for serving the low- 
est-scoring students, who would otherwise have enrolled in the course that had been eliminated. 


MDRC partnered with THECB to examine how colleges implemented the new state guidelines for 
supporting students scoring below ninth-grade proficiency in math on their course placement exams. 
This report presents the findings from a study of a four-week “Bridge Math” course at Houston Com- 
munity College (HCC), which was offered for students who scored between the fourth- and eighth- 
grade levels of math proficiency. This Bridge Math course was intended to boost their basic skills in 
math in order to prepare them to tackle courses at the ninth-grade level — the level at which they 
could eventually enroll in developmental math. 


Starting in fall 2015, students who completed the four-week Bridge Math course had the opportunity 
to immediately enroll in the first course of the developmental math sequence to work toward raising 
their skills to twelfth-grade proficiency. The data and evidence suggest that although some students 
who enrolled in the four-week Bridge course gained the math skills to prepare themselves for the 
developmental-level math sequence, the majority of those students who were referred to the Bridge 
course did not enroll in it, and most who did enroll did not complete it or move on to a developmental- 
level course. 


Colleges that introduce new programs and courses often require more time than expected to fully 
implement them. As found in the field research for this study, students, instructors, and advisors were 
not uniformly aware of the new placement policies and program changes, which contributed to lower- 
than-expected enrollment rates in the Bridge course and still lower transition rates to the developmen- 
tal course for those who passed the Bridge course. Clear vertical and lateral communication among 
the administration, advisors, instructors, and students — as well as through information provided on 
HCC’s website and social media outlets — is particularly critical when a major change in placement 
tules such as this one occurs, affecting so many incoming students’ chances at making it to and through 
developmental math and on to earning their college credential. 
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Chapter 1 


Introduction 


A recent study found that one in three Americans between the ages of 16 and 65 lack the skills 
needed to succeed in middle school math, let alone math at the high school or college level.! 
Encouraged by open admission policies for high school graduates, many of these individuals opt 
to pursue education at community colleges. But before even enrolling in college, most students 
are expected to take high-stakes placement tests that measure their math and English skills, such 
as the ACCUPLACER college placement test. These tests are designed to distinguish between 
students whose skills are below twelfth-grade proficiency levels in math, reading, or writing, and 
those who are deemed “college ready” — that is, students with skills at the twelfth-grade level or 
higher. Nationwide, more than half of community college applicants score below the college- 
ready level; colleges often require these students to pass a series of developmental or remedial 
courses before they can take credit-bearing math or English courses.” In some cases, students 
referred to developmental course sequences may be required to enroll in and pass as many as four 
16-week-long developmental courses before becoming eligible for college-level courses. The ma- 
jority of these students end up dropping out before completing their developmental requirements. 


In recent years, states have undertaken a variety of reforms in response to these low rates 
of success. Some states have simply dropped the requirement for remediation in English or math, 
allowing students to attempt college-level courses regardless of their skill level upon entry to 
college. Other states have shortened the developmental course sequence, dropping one or two 
courses in order to reduce attrition among developmental students. Some states have created dif- 
ferentiated requirements for different majors. For example, students who pursue STEM-related 
majors (science, technology, engineering, and math) may be required to complete college-level 
math courses, whereas students pursuing liberal arts majors may only be required to complete 
eleventh-grade math. In other instances, states or colleges have raised standards for developmen- 
tal courses and routed students who score below the minimum skill level for developmental clas- 
ses to lower-level college-transition courses or programs such as adult literacy programs. 


One state that pursued the strategy of raising the minimum skill level required for devel- 
opmental math was Texas. In 2014, the Texas Higher Education Coordinating Board (THECB) 
implemented major changes in how the lowest-scoring students should be placed and served in 
community colleges. After adopting a new placement test, the state raised the cut score for placing 
students in developmental education and eliminated the lowest-level developmental math course. 
Colleges were then permitted to choose among several options for serving the lowest-scoring 
students, who would otherwise have enrolled in that course. This study places the Texas policy 
in the context of other state reform strategies and examines the response of one Texas college: 
Houston Community College (HCC). In 2015, HCC replaced the lowest level in the sequence of 


‘Goodman et al. (2013). 
Attewell, Lavin, Domina, and Levey (2006). 


developmental math courses with a new four-week “Bridge” course for students who score be- 
tween the fourth- and eighth-grade levels. This Bridge course was intended to boost their basic 
skills in math to prepare them to tackle the ninth-grade level — the level at which they could 
enroll in developmental math.? 


The study was originally intended to be a mixed-methods evaluation, using a regression 
discontinuity design to assess the effects of being referred to the new four-week Bridge course 
compared with being referred to the higher-level developmental math course. Ultimately, how- 
ever, this design proved infeasible as the result of a combination of factors, which are discussed 
in detail in Chapter 3. Instead, this report focuses on interesting findings describing the imple- 
mentation of this new policy. 


Research Questions 


This research study focused on the following questions: 


1. How was the new Bridge course at HCC implemented? The implementation study, 
which relies primarily on field data collected through interviews with students, in- 
structors, and administrators, examined factors that may have affected the effective- 
ness of HCC’s program. These included the extent to which the program components 
were offered as intended, the contrast in experiences of the Bridge students compared 
with students who placed into ninth-grade-level math courses, and the attitudes and 
perceptions of students, instructors, and administrators. 


2. How many students who tested near the ninth-grade proficiency level in math were 
directed to the new Bridge course as a result of the new policy, and what were their 
demographic characteristics? This question aims to clarify the size of the population 
of “affected” students as well as their basic demographic characteristics, such as gen- 
der, race, and English proficiency. The study used student records to address these 
questions. 


3. To what extent did students scoring near the ninth-grade proficiency level in math 
enroll in the college and their assigned math course? Texas administrators and poli- 
cymakers, as well as administrators in other states implementing similar polices, were 
concerned that students whose scores were “below the cutoff’ for developmental 
courses (and hence assigned to a “Bridge” course) would feel discouraged or con- 
fused once they understood that they would have to enroll in a transition program 
before being allowed to enroll in the “regular”? developmental math course. Student 
records were used to measure the likelihood of various enrollment behaviors, and 
whether they differed for students above and below the ninth-grade level. 


>This report uses the term “Bridge” for this course, although the college itself refers to the course by its 
course number, MATH-0106. 


4. How did average academic outcomes among students assigned to the Bridge course 
compare with those of students who were assigned to the developmental math 
course? Were there differences between these two groups in terms of making pro- 
gress through math remediation, attempting college-level math, and earning credits 
toward a college credential? This question is descriptively explored in Chapter 3. 


Sample and Data 


The student sample for the demographics, enrollment, and outcomes analyses includes all stu- 
dents who took the Adult Basic Education math assessment between February 2015 and January 
2016, a total of 5,552 students. The sample is divided into two cohorts based on the date that 
students first took the Texas Success Initiative math assessment exam. The first cohort includes 
3,783 students who first took the math assessment and scored below 336 on the exam between 
February 1, 2015, and August 31, 2015; the second cohort includes 1,769 students who first took 
the assessment between September 1, 2015, and January 31, 2016. Notably, not all students who 
took the test subsequently enrolled at HCC. HCC served as a testing center for nearby high 
schools; consequently, some test-takers may have been high school students who had never in- 
tended to enroll at HCC. 


Data Sources 


Qualitative Data 


The research team visited HCC during the fall 2015 and spring 2016 semesters to learn 
about how the new program was being implemented. During the visits, the team conducted 
individual interviews with administrators and held focus groups with students, advisers, and 
instructors. The purpose of collecting qualitative data was to document how changes were per- 
ceived by the various stakeholders, whether and how students were directed to their assigned 
math programs, and how they experienced their classes if they did enroll. Additionally, the 
research team observed a few Bridge course and developmental math classrooms to try to un- 
derstand the extent to which the new Bridge course was implemented as intended and how the 
experiences of students assigned to this transition program and to the developmental math 
courses differed. The research team also conducted a few post-visit follow-up phone interviews 
with college administrators to gather additional information or gain further clarity about what 
was learned during the visits. 


Interviews were transcribed and analyzed to identify themes and patterns. Participation 
in focus groups and interviews was voluntary and largely self-selected; thus, these relatively 
small groups may not be representative of the opinions and observations of all staff at HCC or 
of all students who participated in the programs described as part of this research. 


Administrative Data 


The analyses of demographics, enrollment, and academic outcomes in this report drew 
upon the following data sources: 


e Student placement test data. These administrative data were provided by HCC 
and the College Board (which developed and scores the placement tests) and 
include information on the assessment results for each student in the sample, 
including scores and test dates. 


e Student demographic data. These data were primarily provided by HCC and 
include information on students’ gender, race/ethnicity, and other selected 
background characteristics. Some demographic data were also provided by the 
College Board. 


e Student transcript data. These administrative data were provided by HCC and 
include information on the courses that students took and the grades that they 
received each semester. 


Overview of the Report 


The next chapter places this study in a broader reform context by describing college transition 
initiatives in other states that are similar to the one implemented at HCC. Although different in 
structure, format, and content, these initiatives in other states were also designed to serve students 
who scored below developmental-level math proficiency. Descriptive findings on the HCC pro- 
gram are reported in Chapter 3. Chapter 4 concludes the report with a summary of key findings 
and policy implications. 


Chapter 2 


New Approaches to Serving Students with Very Low 
Math Skills: A National Perspective 


Across the country, a number of community colleges, college systems, and states have been ex- 
perimenting with implementing a skill-level threshold, or “floor,” for entry into developmental 
education.! Traditionally, most colleges admit all eligible students, even if their math scores are 
at primary school levels. With these new policies, students who score below a certain skill thresh- 
old are directed to different programs or services than the typical sequence of developmental ed- 
ucation courses intended to prepare students for college-level work. Some individual colleges 
have implemented these policies on their own, while, in other cases, state or college systems have 
made recommendations about skill thresholds, leaving individual institutions with discretion over 
whether and how such a policy is implemented. 


As of late fall 2017, some states had implemented statewide policies to reduce the num- 
ber of developmental education levels with the expectation that all colleges within the state 
would comply. In 2017, MDRC released an early report, Raising the Floor, that described the 
programs and services that these states had adopted in an attempt to serve students who tested 
below the minimum threshold. For instance, some state leaders recommended that students be 
referred to and enrolled in other education and workforce programs, such as federally funded 
Adult Basic Education (ABE) programs, while others encouraged colleges to provide short- 
term, non-course-based options, such as online tutorials or multiweek “boot camps,” for build- 
ing these students’ skills. 


While similarities exist in the types of programs that colleges have offered to these low- 
skilled students, the skill-level thresholds set by states and systems and the funding streams to 
support services for low-skilled students differ. For instance, based on interviews with program 
leads in various states, MDRC researchers learned that the skill thresholds set in North Carolina 
and Virginia allowed students with eighth-grade skill levels to enroll in their developmental math 
course modules. Students at these skill levels would generally start at the first module and then try 
to make progress through 8 or 10 modules before being allowed to enroll in college-level classes. 


In contrast, Colorado, Connecticut, and Texas (the state where the current study was con- 
ducted) have set the threshold at ninth-grade skill levels; thus, any student scoring at a ninth-grade 
level but below the twelfth-grade level could enroll in developmental math courses, while those 
with eighth-grade level skills or below were redirected away from developmental courses and into 
other programs or courses. Given these higher skill-level cutoffs for entrance into developmental 
courses, it is likely these policies affected far more students in Colorado and Connecticut than 


‘Similar policies were also enacted in these states for reading and writing; however, only developmental 
math policies are discussed in depth here, since that is the focus of this report. 


Kansas and Missouri are two examples; see Developmental Education Working Group (2014) and Mis- 
souri Department of Higher Education (2017). 


3Visher et al. (2017). 


in Virginia and North Carolina. (The policies in Texas are discussed in more detail below.) For 
instance, in 2008-2009, Colorado estimated that 7,100 of its 24,000 developmental students 
were testing at the lowest level of developmental math. As such, as much as 33 percent of 
students entering community colleges in Colorado may be redirected into college-transition 
programs or courses. 


As they revised their developmental education policies and established minimum skill 
thresholds for entry, Colorado’s and Connecticut’s college systems (and both states) also changed 
the number and types of developmental courses that could be offered. In both states, colleges 
within these systems were limited to a single, one-semester developmental math course.* Students 
whose test scores placed them above developmental level were to be enrolled in college-level 
courses. For students with test scores below the developmental level, state leaders recommended 
that they be offered transition or pre-enrollment services either at the college or at a local Adult 
Basic Education (ABE) or community program. 


Regardless of their location, these transition services were not considered college classes 
in Colorado and Connecticut, were not part of the statewide college course catalogue, and did not 
offer college credit that could count toward a credential or toward eligibility for federal financial 
aid.° Also, funding for these programs and services had to rely on sources other than or in addition 
to funding streams from the college — for example, workforce development or adult education. 


Leaders in Colorado and Connecticut noted that the key rationale for these changes was 
the very low completion rates of students in the lowest-level developmental education courses. 
As one leader in Colorado stated, students in the lowest-level developmental math class “had less 
than a 5 percent chance of enrolling in and passing a college-level math course.’ Leaders in both 
states also noted that many of these low-scoring students were using up their financial aid in 
courses where they were unlikely to be successful (that is, developmental math course sequences) 
and wanted to limit this practice. In other words, the assumption was that since students with such 
low skills were unlikely to pass developmental courses, the cost was greater than the potential 
benefit of allowing these students to enroll. 


In contrast, Virginia and North Carolina divided their traditional developmental math 
courses into modules, with students being placed into various modules based on their scores on a 
state-based diagnostic exam. In North Carolina, students took a diagnostic assessment for all six 
of the developmental modules and received a score between 0 and 7 for each module. Any student 
who failed module one and tested below a 7 on every other module was recommended to take 
remedial courses. State leaders recommended that students take the Test of Adult Basic Education 
(TABE) or the Comprehensive Adult Student Assessment System (CASAS) test — both tests 
used for placement in the nation’s federally funded adult education programs — to determine 


4Visher et al. (2017); Michael and McKay (2015). 


These services or programs did not offer developmental credits, which were generally nontransferrable 
credits that did not count toward a two-year or four-year degree, nor college-level credit courses, which generally 
are transferable and count toward a degree. 


Personal interview, November 2017. 


their skill levels. After taking a basic skills support course, students would retake the TABE or 
CASAS and had to demonstrate an increase in one grade level of skills to enroll in the develop- 
mental math modules. North Carolina students could also retake the developmental education 
diagnostic exam to try to place out of additional developmental course modules. 


In Virginia, students take the Virginia Placement Test to determine which of nine differ- 
ent modules they should be placed in. If students test below module 1, they are referred to a 
remedial course, which teaches math with whole numbers, or about a third-grade skill level. One 
to two Basic Skills courses were generally offered at larger community colleges across the state; 
however, smaller campuses generally did not have enough students to support a full course and 
thus would create different support options for these students. 


Tracking Low-Scoring Students’ Progress in College-Transition 
Services 


It is difficult to document the numbers of incoming students who are affected by the policy and 
program changes described above, although a study of several dozen community colleges in five 
states sheds some light on the question. This study tracked first-time, credential-seeking students 
who enrolled in college from fall 2003 to fall 2004 using a database of 57 colleges in nine states.’ 
This analysis revealed that approximately 33 percent of entering students tested into a develop- 
mental math class that was three or more levels below college-ready,*® and another 18 percent 
tested two levels below.’ As such, colleges and states that set a ninth-grade skill threshold for 
entry into developmental math may be affecting as much as one-third of their incoming commu- 
nity college population. 


Little state or college data exist to track students’ entry into college-transition programs, 
their progress through these programs, and their subsequent reentry into developmental or col- 
lege-level math. This is particularly the case for transition programs that are offered and funded 
by funding streams outside the normal community college sphere. Workforce programs and ABE 
programs have different accountability systems and date-reporting procedures than community 
colleges. Therefore, the data that are used in ABE programs are generally not easily accessible to 
colleges and vice versa, making it difficult to track students who may transition across these sys- 
tems. Program administrators who were interviewed in each of the states profiled in this report 
also noted that no systems currently exist for tracking lower-skilled students referred to alternative 
programming, even when those programs or services are located at the college. 


’The nine states were Connecticut, Florida, Ohio, New Mexico, North Carolina, Pennsylvania, Texas, Vir- 
ginia, and Washington. 

’In Bailey, Jeong, and Cho (2010), this percentage refers to students attending colleges that offer courses 
three levels or more below college-ready. However, 18 colleges (of the 57 in this study) do not offer courses 
three levels or more below college-ready, so the number of students assigned to courses three levels or more 
below college-ready is lower or higher than 33 percent. 


*Bailey, Jeong, and Cho (2010). 


While little reliable data exist, MDRC’s interviews and review of online reports on Con- 
necticut’s and Colorado’s revisions confirm that significant numbers of students are affected by 
policies that reroute low-scoring students to alternative services and programs. For example, in a 
2015 report on Colorado’s developmental education redesign, at least 12 of the state’s 13 com- 
munity colleges had created services for low-skilled students on their campuses or made connec- 
tions with local ABE or workforce centers to develop programs for these students off-site.'° Sim- 
ilarly, a 2014 report analyzing Connecticut community colleges’ developmental education 
reforms noted that at least half of the state’s 12 colleges had implemented transition programs for 
lower-skilled students. '' However, little is known about how many affected students actually en- 
roll in or complete these programs. 


Some of these colleges have reported results from these pilots in descriptive studies. For 
example, one community college in Connecticut noted that 64 percent or more of the students 
who completed their boot camps had increased their developmental education placement scores 
by at least one developmental level.'? Neither Virginia nor North Carolina have done in-depth 
studies of college-transition programs for low-skilled students, though Virginia noted overall in- 
creases in enrollment and completion of gateway college-level math courses after implementing 
the math modules revisions. !7 


To summarize, little is known about how these transition programs serving low-scoring 
students are operating, how many students are affected, and, of those, how many are referred to 
and enroll in them. Even less is known about how such students fare compared with how they 
might have fared if allowed to enroll directly in developmental courses. This report presents find- 
ings from a study of one such program operated at Houston Community College (HCC) in Texas. 


The Texas Way 


In 2014, the Texas Higher Education Coordinating Board (THECB) released guidelines as part 
of the Texas Success Initiative (TSI) to encourage the 22 community colleges in the state to re- 
vamp their assessment and placement practices for incoming students. THECB is a state agency 
that reports to the state legislature and that oversees all two-year and four-year postsecondary 
institutions in Texas, as well as adult education programs. Colleges are required to submit student 
data reports to THECB regularly. As is the case in many other states, THECB can influence policy 
and practice, but its ability to mandate policy is limited. As one state official noted, there is often 
considerable confusion among local institutions about what is a mandate and what is not — as 
was the case for the TSI passed by Texas legislators in 2013. In the end, virtually all the commu- 
nity colleges in Texas did implement significant changes to their assessment, placement, and 


‘OMichael and McKay (2015). 
"Senserrich (2014). 
'2Senserrich (2014). 


‘Personal communication with Virginia’s Community College Office of Institutional Research and Effec- 
tiveness, 2014. 


education of low-scoring students in response to the TSI, although few of these changes were 
mandated by the state. 


The TSI included both mandates and recommendations that had the potential to lead to 
fundamental changes in how colleges would assess incoming students and which services should 
be offered, depending on their proficiency levels in both math and English. The new assessment 
and placement guidelines would lead to programs that replaced much of the old system, which 
sorted students with precollege math skills into as many as four different developmental courses 
depending on the results of their math assessment. The new program led colleges to (1) eliminate 
the lowest level of developmental math courses, which served students with skills below a ninth- 
grade level, some with skills below a fourth-grade level; (2) add a new assessment, intended to 
be more fine-tuned; and (3) use a standardized cut score to sort students into three groups: those 
ready for college-level math, those with ninth- to eleventh-grade proficiency, and those with be- 
low ninth-grade proficiency. 


Two elements of the TSI were mandated. First, colleges were mandated to use a new 
assessment, the TSI Assessment (TSIA), that was designed to more accurately assess students’ 
math, reading, and writing skills. TSIA replaced the four tests used by community colleges in 
Texas at the time. Second, the TSI mandated a specific cut score on the TSIA — dividing students 
into college-ready and not-college-ready groups. In the past, colleges could determine their own 
cut scores, resulting in wide variation in how college readiness was defined. In 2015, the college- 
ready cut score was 350. Students who scored below 336 on the TSIA, therefore placing far below 
college level, were given an additional set of questions called the “TSI Adult Basic Education 
(ABE) Diagnostic Assessment” (hereafter referred to as the “ABE test’). This test then divided 
students into six different sublevels, ABE Level 1 through ABE Level 6.'* 


The state recommended that colleges choose from a list of offerings for students depend- 
ing on their ABE Levels as follows: 


e Students in ABE Levels | and 2, equivalent to third-grade proficiency or be- 
low, should be redirected from enrolling in developmental education and, in- 
stead, be referred to zero or noncredit courses, Adult Basic Education, Contin- 
uing Education, or other programs. 


e ABE Levels 3 and 4, equivalent to fourth-grade to eighth-grade proficiency, 
should be redirected from enrolling in developmental education and instead be 
referred to transition programs or courses such as Bridge. 


“The term “ABE” is not necessarily accurate, since neither the test nor the related programs are necessarily 
supported by Adult Basic Education funding. Although developmental education courses and ABE courses both 
serve low-skilled students, ABE programs have traditionally been reserved for those without a high school di- 
ploma and generally have differing funding streams, regulations, and accountability systems. THECB adopted 
this terminology during the early phases of formulating the policy, and its use persisted despite some resistance 
and some confusion. This report uses the term to be consistent with state and local familiarity with it in associa- 
tion with the policy and program changes that are the subject of this study. 


e ABE Levels 5 and 6, equivalent to ninth- to eleventh-grade proficiency, were 
to be referred to developmental education courses. !° 


The TSI, as implemented in most of the Texas community colleges, amounted to “raising 
the floor” for entry into regular developmental education, mostly because colleges eliminated the 
lowest-level course in the developmental math course sequence and only students with ninth- 
grade skills or higher were eligible to take the shortened developmental course sequence that 
remained. In the past, students whose skills were below eighth-grade levels could still enroll in 
up to four developmental courses. Colleges began implementing the recommendations as early 
as 2013, with most launching their programs by fall 2015. 


'STexas Higher Education Coordinating Board (2014). 
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Chapter 3 


Findings for the Houston Community College Program 


This chapter begins with context on Houston Community College (HCC) and its four-week math 
“Bridge” course, including a description of the intended design of the course and how it was 
implemented during the study period. Students with Adult Basic Education (ABE) scores at Level 
3 and Level 4 were referred to the Bridge course; those with placement scores at Level 5 and 
Level 6 were referred to Foundations of Math — the first full developmental math course. This 
chapter presents descriptive data on the numbers, characteristics, and outcomes of students whose 
first test scores were directly below or at ninth-grade proficiency (Levels 4 and 5, respectively). 
Results in this chapter are presented according to the test scores students received the first time 
they took the test.! 


A Brief Aside 


As noted earlier, in addition to the descriptive analyses discussed below, this study intended to 
estimate the effect of being referred to the Bridge course compared with being referred directly 
to developmental math. A strong research design, known as a regression discontinuity design 
(RDD), was going to be used to for these analyses. However, RDD relies on several important, 
empirically testable assumptions. In this instance, these assumptions were violated. 


First, a wide range of students took the placement test at HCC regardless of whether they 
intended to enroll at HCC or not. Many students who took the placement test did not go on to 
enroll at HCC; thus they were never exposed to the intended program (the Bridge course or Foun- 
dations of Math, depending on their test level). In addition, even among students who did enroll 
at HCC, many did not enroll in their intended math class. This happened either because a student 
didn’t enroll in any math course or because the student enrolled in a math class different from the 
one indicated by the initial placement test score. Collectively, these factors reduced the service 
contrast between students on either side of the placement cut point. Additionally, some infor- 
mation was missing in available administrative records, with data being more likely to be missing 
for students who did not enroll at HCC.’ 


Background 


The Houston Community College system consists of six colleges and 21 campuses that serve the 
greater Houston area. One of the largest community colleges in the country, HCC had more than 


'Eleven percent of students retested and received a higher score prior to or during spring 2016. Not all these 
retest scores were high enough to facilitate skipping levels. 


Additional details regarding the RDD analyses and validity tests that were conducted are available upon 
request by e-mailing angela.boatman@vanderbilt.edu. 
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100,000 college and workforce students enrolled in the 2015-2016 academic year. Approximately 
one-third of HCC’s students are Hispanic and one-third are African American, with the remaining 
students a mix of white, Asian, and other ethnicities. HCC serves mostly low-income students.* 


The placement policy for assignment into Bridge and other HCC math courses during the 
years of this study is shown in Figure 3.1. In their first semester of enrollment at HCC, students 
with ABE placement scores at Levels | and 2 were referred to career technical education courses, 
typically offered off-campus for zero credit. Those placing at Levels 3 and 4 were referred to the 
Bridge course; those at Levels 5 and 6 were referred to the first full-length developmental course, 
Foundations of Math. HCC offered two courses in its developmental math sequence, Foundations 
of Math and Developmental Algebra; however, only the former was required for all students who 
received an ABE score. Given that not all students were required to complete the second devel- 
opmental math course,‘ this report focuses on comparing the ABE test-takers who were referred 
to the Bridge course with those who were referred directly to the Foundations of Math course. 


The intended enrollment sequence for students below ninth-grade proficiency (Levels 3 
and 4) was to first take the four-week Bridge course for one developmental credit at the start of 
the semester to sharpen their basic math skills. The content taught in the Bridge course was in- 
tended to reinforce basic math concepts normally taught in elementary and middle school, such 
as fractions, decimals, and percentages. These students could then enroll in an eight-week section 
of the Foundations of Math course during the second half of the same semester or could wait until 
the following semester to enroll in Foundations of Math. A passing grade on the final exam of the 
four-week course qualified students to enroll in Foundations of Math, where they were to apply 
their learned math skills to solve arithmetic operations using numbers in various formats — basic 
geometry, polynomials, and linear equations. Students who did not pass the Bridge course during 
the first four weeks of the semester could take it again in the next four weeks and still have a 
chance to enroll in an eight-week section of Foundations of Math starting in the second half of 
the semester as long as seats were available within those sections. In contrast, the intended enroll- 
ment sequence for students with ABE placement scores at Levels 5 and 6 was to enroll directly 
in Foundations of Math at the start of the semester. 


Distinctions Between Bridge Math and Foundations of Math 


Several distinctions between the two courses should be noted. First, most Bridge sections were 
offered two days per week, for two hours per class, for a total of 16 hours of instructional time. A 


Data from the 2015-2016 academic year reported to the Texas Higher Education Coordinating Board as 
part of the Perkins Basic Grant Program showed that 81 percent of students were economically disadvantaged. 
This percentage was calculated using various factors, including students’ annual income, eligibility for food 
stamps or certain other public assistance programs, or receipt of a Pell Grant. 

‘Developmental Algebra was an additional requirement only for students with science, technology, engi- 
neering, and math (STEM) majors. 
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few sections had a four-hour, once-a-week format. In contrast, the Foundations of Math course 
offered 64 hours of instructional time, whether the section lasted eight weeks or a full semester. 
Thus, if students referred to the Bridge course followed the intended path (Bridge Math followed 
immediately by Foundations of Math), they would complete 80 hours (16 in Bridge and 64 in 
Foundations) of instructional time in a single semester — 16 more hours than students referred 
directly to Foundations of Math. 


Second, Bridge students had the opportunity to retake the course in the same semester if 
they failed it, whereas students who failed Foundations of Math typically had to wait until the 
following semester to retake their course (unless they failed the first eight-week session and chose 
to enroll in the second eight-week session in the same semester, which very few students did). 
Moreover, while the Bridge course did not have attendance requirements, regular attendance in 
Foundations of Math was required and counted toward a student’s final grade for the course. 


Third, Bridge course instructors designed their own content materials, lectures, and class- 
room activities as they saw fit within the four-week period, leading perhaps to greater variation 
in instruction. Much like the Bridge course, sections of the Foundations of Math course were 
largely taught by adjunct instructors in the Developmental Math department. Although no stand- 
ard textbooks or syllabi were uniformly used across the Bridge course sections, some instructors 
worked together to share worksheet packets, online videos, and other instructional materials. 
Meanwhile, Foundations of Math instructors generally followed the same textbook and pacing 
across all sections — an online learning software (MyMathlab) supplemented course lectures and 
was typically utilized across the course sections of Foundations. 


The paragraphs above describe the typical experience of students enrolled in Bridge or 
Foundations of Math. However, as is covered in more detail below, many students did not enroll 
in their initially assigned courses. Moreover, the small subset of students assigned to the Bridge 
course who took and passed it rarely went on to take the Foundations course in the same semester. 


Given its shortened course length, the Bridge course alone, as it was most commonly 
experienced, offered substantially less “seat time” than the Foundations of Math course. That 
noted, a student who took the Bridge course and Foundations of Math within the same semester 
would have substantially more seat time than a student who only enrolled in Foundations of Math, 
which just didn’t happen that often. 


With this background in mind, this research sought to answer two questions: 


1. To what extent did students who were referred to Bridge or Foundations of Math 
enroll in their assigned math course during the 2015-2016 academic year? 


2. Did test-takers referred to the Bridge course pass Foundations of Math and enroll in 
subsequent math courses at different rates than test-takers assigned directly to Foun- 
dations of Math during the 2015-2016 academic year? 


The first question is of interest because for the referral policies to have a chance of achiev- 
ing their intended effects, students must comply with the referrals. If there is compliance, then 
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understanding the effects of these different placements is important. If compliance is very low, 
then the institution may want to consider other options for how to proceed. 


The second question does not attempt to get at the effect of being referred to Bridge com- 
pared with being referred to Foundations of Math. It simply attempts to describe any differences 
in outcomes between the two groups with different scores. Such differences may be related to or 
caused by a variety of factors, including variations in the student populations receiving higher or 
lower test scores. Examination of any differences may provide the institution with more insights 
for future decisions on placement and assessment practices, particularly for math. 


Do Students Scoring at Level 4 Enroll in Bridge? 


Most HCC students referred to the Bridge course did not comply with their referral. HCC de- 
signed the Bridge course to be completed in four weeks so that students could then enroll in the 
Foundations of Math course within the last eight weeks of the same term. Among HCC enrollees 
who were first scoring at Level 4, 25 percent attempted Bridge as their only math course in their 
first semester after referral. Even fewer, 9 percent, took both the Bridge Course and Foundations 
of Math (the intended sequence) in their first semester after referral. According to interviews with 
HCC administrators and staff, they faced several implementation challenges that may have con- 
tributed to fewer students moving from the Bridge course to Foundations of Math in the same 
semester: 


1. Notall HCC academic advisers were fully aware of the expectation that students who 
passed a Bridge course were expected to immediately enroll in Foundations of Math 
in the same semester. 


2. Registration in these second-half semester courses was logistically difficult, as the 
automated enrollment systems sometimes would not allow automatic enrollment for 
such students, requiring students to seek help from their advisers. 


3. Some students who passed the Bridge course reported not being able to enroll in the 
Foundations of Math course because they could not find an open section later in the 
semester or could not make the open sections fit with the rest of their class or work 
schedules. 


4. Instructors sometimes failed to inform students of what they needed to do to enroll 
in the Foundations of Math course. 


To address problems associated with enrolling in Bridge and Foundations consecu- 
tively, HCC’s math department made changes to its spring 2016 course scheduling so that the 
Bridge course and the Foundations of Math course sections would be taught back-to-back by 
the same instructor during the same semester. However, the rates of enrolling in Bridge and 
Foundations together were similar across the fall 2015 and spring 2016 cohorts, as shown in 
Appendix Table A.1. 
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Not surprisingly, some Bridge course instructors who were interviewed said that they 
struggled to fit all of the necessary content into a compressed, four-week course. Some instructors 
noted that many of their students quickly accumulated absences early in the course, inevitably 
making them fall behind with very little time to catch up. Students who participated in focus 
groups reported varying experiences with the content, with some finding the course too easy and 
boring, while others finding it too hard — highlighting the challenge of teaching a course to stu- 
dents whose math abilities range from fourth-grade to eighth-grade proficiency. 


Numbers and Characteristics of ABE Test-Takers 


In this section, student records are used to document the numbers, characteristics, and math course 
and college enrollment patterns of students whose first test scores indicated proficiency levels at 
and below the ninth-grade level. All students who applied to the college and did not qualify for 
waivers or exemptions were required to take the Texas Success Initiative (TSI); students who re- 
ceived low enough scores were automatically administered an additional assessment, the ABE test, 
to determine a more precise grade-level proficiency. As mentioned above, students whose math 
skills were assessed at third grade or below (Levels 1 and 2) were to be referred to zero- or low- 
credit courses, including workforce programs typically offered at a workforce training site located 
off-campus. Students whose math skills were assessed between fourth and eighth grade (Levels 3 
and 4) were referred to the Bridge course. Students whose scores indicated proficiency between 
ninth and twelfth grade (Levels 5 and 6) were to be referred to the Foundations of Math course. 
Some retesting was done, but only 7 percent of students retested and received higher scores prior 
to their first expected enrollment semester (with 11 percent retesting through spring 2016), and not 
all of these scores were high enough to place students into higher-level math classes. 


Approximately 14,000 individuals first took the TSIA during the time frame for this 
study. From this group, 5,552 — approximately 40 percent — scored far enough below the “col- 
lege-ready math skills” cutoff that they were then administered the ABE Diagnostic.* The distri- 
bution of ABE Diagnostic scores is shown in Figure 3.2. Of these 5,552 students who took the 
ABE assessment, 2,628 scored below ninth-grade level, which corresponds to an ABE Level 4 
or lower, indicating below developmental math level. These 2,628 students represent just under 
20 percent of the total number of students who took the TSIA and just under half of the students 
who took the ABE. In other words, one-fifth of all TSIA test-takers were affected by the new 
assessment and placement policies by achieving a score that placed them below the cut point for 
developmental math. 


As a reminder, this study examines differences in college enrollment and math course- 
taking behavior among ABE test-takers who first scored at Level 4 and 5, where most scores are 
distributed. As shown in Table 3.1, of the full sample, 43 percent of Level 4 and 5 test-takers were 


Scores ranged from 310 to 390. The college cutoff was 350; students who scored 335 or below were re- 
quired to take the ABE Diagnostic. 
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Figure 3.2 
Math Test-Takers, by ABE Level, Houston Community College (HCC) 
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SOURCE: Calculations from placement data from Houston Community College and the College Board. 
Derived from data provided by the College Board. Copyright © 2013-2019 The College Board. 
(www.collegeboard.org) 


NOTES: This sample includes all individuals with a first reported TSIA (Texas Success Initiative Assesment) 
score below 336 between February 1, 2015, and January 31, 2016. A total of 13,957 people took the TSIA 
during this window. Those scoring below a 336 were required to take the ABE test. A total of 5,552 
(approximately 40 percent) out of 13,957 TSIA test-takers scored below a 336, thus also taking the ABE test. 
The percentages above each bar capture the percentage of students at each ABE level. Data include an 
individual's first TSIA and ABE score only, and includes all test-takers at HCC, regardless of subsequent 
enrollment. 


female, with 32 percent missing gender data; the remaining students (approximately 25 percent) 
were male. This large gender gap may reflect differences in “missingness” by gender or also may 
reflect some national trends where females enroll in college at higher rates than males.° Reflecting 


® Lopez and Gonzalez-Barrera (2014). 
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Table 3.1 


Demographic Characteristics of ABE Test-Takers, 
Houston Community College 


All ABE Level 4 

and Level 5 
Characteristic Test-Takers (1) Level 4 (2) Level 5 (3) 
Number of test-takers 4,976 2,204 2,772 
Female 43% 42% 43% 
Gender missing 32% 36% 28% 
White 6% 4% 7% 
Black 31% 35% 27% 
Hispanic 25% 17% 31% 
Other race 6% 7% 5% 
Race missing 32% 37% 29% 
Low English proficiency 5% 8% 3% 
English proficiency missing 33% 37% 30% 
Single parent 10% 11% 9% 
Single parent missing 33% 37% 30% 
Deiat pies or more of high 34% 30% 37% 
Years of high school math 48% 52% 45% 


missing 


SOURCE: Data on gender, race, English proficiency, and single-parent status are from 
Houston Community College demographic datasets. Some race data and all data on high 
school math experience are self-reported from the College Board’s placement data. 
Derived from data provided by the College Board. Copyright © 2013-2019 The College 


Board( www.collegeboard.orq). 


NOTES: Each cell presents the percentage of ABE test-takers. Data are pooled across 
cohorts 1 and 2, which includes all individuals with a first reported ABE score of Level 
4 or Level 5 between February 1, 2015, and January 31, 2016. 


the diversity of HCC, a total of 31 percent of Level 4 and 5 test-takers identified as black, 25 
percent as Hispanic, 6 percent as white, 6 percent as another race, and again 32 percent did not 
report a race. Five percent of test-takers had low English language proficiency, and 10 percent 
were single parents, according to school records; for one-third, data on English and single-parent 
status were missing. Notably, there are differences in missing data across the two levels of ABE 
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test-takers, shown in Table 3.1, which may be due to Level 5 students enrolling more often and, 
therefore, having more complete demographic information.’ 


Course Enrollment Behavior 


Table 3.2 describes the college and math course enrollment behavior for students who first scored 
at ABE Levels 4 and 5 at three different points in time within the study time frame: at the end of 
the first semester after taking the ABE assessment, the end of the second semester after taking the 
assessment, and by the end of the fourth semester.* The first two columns apply to Level 4 and 5 
ABE test-takers regardless of whether they actually enrolled at HCC. Columns 3 and 4 apply only 
to those students who enrolled in their first expected semester. 


Unsurprisingly, students first scoring at ABE Level 4 were less likely to be enrolled at 
HCC, less likely to enroll in math at all, and less likely to enroll in their assigned math class than 
Level 5 students. This was true in the first semester and also up to two years after first taking the 
test. For example, columns | and 2 of panel 2 show that 40 percent of test-takers first scoring at 
ABE Level 4 and 33 percent of those first scoring at ABE Level 5 still had not enrolled at HCC 
by the end of the second semester after taking the ABE assessment. By the end of four semesters, 
the gap between students who originally scored at Level 4 and Level 5 is still apparent, with 35 
percent of Level 4 and 29 percent of Level 5 test-takers still not enrolling at HCC. The high 
number of ABE test-takers who do not enroll at HCC is due, in part, to many high school students 
taking the test for diagnostic purposes and who may or may not have ever intended to enroll at 
HCC. Some of this drop-off also likely represents students who were interested in HCC (or col- 
lege more generally) but still failed to enroll. 


To better understand why test-takers decided ultimately not to enroll at HCC, MDRC 
administered a survey to low-scoring students who had enrolled in at least one ABE or continuing 
education class but decided not to reenroll at HCC for the fall 2016 semester, asking students to 
explain their reasons for not enrolling. Among students who responded to the survey, most cited 
financial reasons, the need to work, or family responsibilities. However, 15 percent of students 
cited reasons related to their math test scores and course assignments. Results from the survey 
need to be viewed with caution, however, as the response rate was low — under 10 percent. 


Percentages reported here differ somewhat from the descriptive statistics reported in the last report (Visher 
et al., 2017) because the sample here was defined more narrowly to identify students for inclusion in the regres- 
sion discontinuity analysis; gender, English proficiency, and single-parent percentages were calculated among 
non-missing values only in the earlier publication. 

8A student’s first semester after taking the ABE assessment is fall 2015 for Cohort 1 and spring 2016 for 
Cohort 2. For Cohort | the end of the second semester is summer 2016. For Cohort 2 it is fall 2016. The end of 
the fourth semester is summer 2017 for Cohort | and fall 2017 for Cohort 2. 
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Table 3.2 


College and Course Enrollment Behavior of ABE Test-Takers After One 


Semester, Two Semesters, and Four Semesters, Pooled Cohorts, 
Houston Community College (HCC) 


All Test-Takers Enrolled Students 
Behavior Level4(1) Level 5 (2) Level 4(3) Level 5 (4) 
Number of ABE test-takers 2,204 2,/72 1,046 1,444 
(1) End of first semester after ABE assessment (%) 
Did not enroll at HCC 53 48 —_ —_— 
Enrolled at HCC, did not take any math 2 22 55 41 
Enrolled, took Bridge only 12 2 25 3 
Enrolled, took Foundations only 5 25 10 48 
Enrolled, took both Bridge and Foundations 4 1 9 3 
Enrolled, took other developmental math 0 1 1 2 
Enrolled, took college-level math 0 1 1 2 
(2) By end of second semester after ABE assessment (%) 
Did not enroll at HCC 40 33 — — 
Enrolled at HCC, did not take any math 25 21 40 29 
Enrolled, took Bridge 26 5 45 8 
Enrolled, took Foundations 18 38 32 62 
Enrolled, took other developmental math 5 12 8 19 
Enrolled, took college-level math 2 4 2 6 
(3) By end of fourth semester after ABE assessment (%) 
Did not enroll at HCC 35 29 —_— _— 
Enrolled at HCC, did not take any math 21 18 32 23 
Enrolled, took Bridge 32 7 51 10 
Enrolled, took Foundations 26 45 42 68 
Enrolled, took other developmental math 10 21 18 29 
Enrolled, took college-level math 6 11 9 17 


SOURCE: Calculations from transcript and placement data from Houston Community College and placement data from 


the College Board. Derived from data provided by the College Board. Copyright © 2013-2019 The College Board 
(www.collegeboard.org). 


NOTES: Cohorts 1 and 2 are pooled. A student's first semester after taking the ABE assessment is the fall of 2015 for 
Cohort 1 and the spring of 2016 for Cohort 2. For Cohort 1 the end of the second semester is the summer of 2016. For 
Cohort 2 it is the fall of 2016. The end of the fourth semester is summer of 2017 for Cohort 1 and the fall of 2017 for 
Cohort 2. “Other developmental math” includes the second course in the developmental math sequence, Developmen- 
tal Algebra, which was required only for students with STEM majors. The percentages in panels 2 and 3 do not sum to 
100, as they capture enrollment outcomes across multiple semesters, over which period many students enrolled in 
more than one class. The sample for columns 3 and 4 remains constant across panels; students who did not enroll in 


their first expected semester are excluded from panels 2 and 3, even if they enrolled later. 


Many enrolled students delayed taking math in their first semester. Approximately one- 
fourth of test-takers in both groups (26 percent for Level 4 and 22 percent for Level 5) enrolled 
at HCC in the first semester but did not enroll in a math course. This rate of delaying taking math 
is even more striking when one considers that 55 percent of enrolled Level 4 students and 41 
percent of enrolled Level 5 students put off taking math (columns 3 and 4). If completing reme- 
diation in math early is important, as some argue, these numbers are cause for concern. 


After four semesters, many students still had not enrolled in the math class associated 
with their ABE score or enrolled in math at all, with even more opting out among those at Level 
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4. By the end of four semesters, only 32 percent of Level 4 test-takers had enrolled in the Bridge 
course (column 1) and 45 percent of all Level 5 ABE test-takers had enrolled in the Foundations 
of Math course (column 2). If this sample is restricted to just those students who enrolled at HCC 
(columns 3 and 4), the gap between ABE Levels 4 and 5 students remains large. Approximately 
half (51 percent) of students at ABE Level 4 had enrolled in the Bridge course by the fourth 
semester, while 68 percent of the Level 5 students had enrolled in the Foundations of Math course. 
These results are consistent with an emerging pattern: ABE Level 4 students were less likely to 
enroll in math and specifically in the course associated with their ABE score than Level 5 students 
in their first semester, but enrollment was low for both groups.’ 


Among enrolled students in the sample, few had enrolled in college-level math by the 
end of four semesters. As expected, ABE Level 5 students were ahead of ABE Level 4 students, 
with 17 percent having enrolled in a college-level math course by the end of four semesters, com- 
pared with 9 percent of ABE Level 4 students. As described above, HCC administrators who 
were interviewed stated that students would pass the four-week Bridge course at the start of their 
first semester and then immediately enroll in Foundations of Math within the same semester. 
Table 3.2 shows that this successive enrollment pattern rarely happened. Only 9 percent of en- 
rolled Level 4 students passed the course in their first semester and went on to enroll in the Foun- 
dations of Math course in the same semester. Interestingly, a total of 6 percent of enrolled Level 
5 students took the Bridge course in the first semester, despite not being formally assigned to that 
course. Approximately half of these students enrolled in Bridge alone, and half enrolled in Bridge 
and Foundations together (as shown in Table 3.2, column 4). Findings from interviews and focus 
groups suggest that some students may have wanted a refresher course in basic math prior to 
taking the first developmental course or that they ended up enrolling in the course because there 
may have been confusion in advising. 


The low number of students who transitioned as expected from the Bridge course into 
the Foundations course may be due to communication and procedural challenges, resulting in 
students not being aware of the opportunity or unable to find an available Foundations course 
section with open seats. This is corroborated by the field research, as HCC administrators who 
were interviewed described the substantial organizational restructuring that the college undertook 
during the first semester the Bridge course was offered. This restructuring included a district-wide 
change to split up the administration of the math department and developmental education, which 
resulted in some initial miscommunication and misunderstandings about the Bridge course and 
delays in assigning instructors to teach it. This also caused a temporary shortage of space in the 
Bridge course, a situation that administrators tried to remedy in the second semester by better 
aligning the sections of Bridge and Foundations courses. 


°The Level 4 and Level 5 students received higher scores on retests at the same rate (7 percent during the 
period prior to their first expected semester, and 11 percent through spring 2016), so retesting does not explain 
why the Level 4 students were less likely to enroll in their originally assigned course. 
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Focus groups with students and instructors indicated that some students were not enthu- 
siastic about being assigned to the Bridge course, which could help explain low enrollment. '° 
According to a few academic advisers, some students felt embarrassed to be at such a low math 
level and wanted to retest and wait to take math until they placed at a higher level, which some 
advisers encouraged them to do. 


To summarize, four semesters after taking the ABE math assessment, ABE Level 4 stu- 
dents were still well behind ABE Level 5 students in terms of taking math courses. Among stu- 
dents who enrolled in their first semester after the test (fall 2015 for Cohort 1 and spring 2016 for 
Cohort 2), 68 percent of those assigned to the Foundations of Math course (ABE Level 5) had 
enrolled in this course after two years, compared with only 42 percent of those assigned to Bridge 
math (ABE Level 4). Even after four semesters, the Bridge course students had, on average, failed 
to catch up to the students assigned to Foundations of Math. Of course, assuming that the ABE 
assessment accurately reflects math abilities, the Level 4 test-takers were, by definition, less well 
prepared to succeed in math than those with Level 5 scores. Nonetheless, these data reveal that 
many students are making a priority of other courses over beginning to address their math re- 
quirements. 


Table 3.3 examines the course enrollment and math course completion rates for students 
who enrolled for at least one semester at HCC and assesses outcomes after four semesters. '! The 
two columns compare students who achieved Level 4 scores with those who achieved Level 5 
scores. Among Level 4 students (column 1), 50 percent ultimately enrolled in the Bridge course 
by the end of four semesters. However, only 32 percent of Level 4 students eventually passed the 
Bridge course within that time frame. The pass rate among those who took Bridge was just 64 
percent. Among all Level 4 students, 40 percent (567 students) eventually made it to the Founda- 
tions of Math course within four semesters, 35 percent (200 students) of whom enrolled in Foun- 
dations of Math — having skipped the Bridge course. Only 9 percent eventually enrolled in a 
college-level math course by that time. By the end of four semesters, 21 percent of all Level 4 
students had passed Foundations of Math and 6 percent had passed a college-level math course. 
As mentioned previously, HCC offered one other developmental math course: Developmental 
Algebra, which was required for STEM (science, technology, engineering, math) majors after 
first completing Foundations of Math. Conditional on completing Foundations of Math, 65 per- 
cent of Level 4 students went on to enroll in Developmental Algebra. 


As expected, Level 5 students progressed further through the math sequence over the 
course of four semesters, though their persistence through the math sequence was still low. 
Among Level 5 students (column 2 of Table 3.3), 63 percent had enrolled in Foundations of Math 


'OMDRC researchers attempted to recruit HCC students eligible for Bridge but who did not enroll in the 
course in order to better understand their reasons for not enrolling. However, none from this group agreed to 
participate in focus groups. 

"Panels 3 and 4 of Table 3.2 follow students who enrolled in the first expected semester and remained 
through four semesters. In contrast, Table 3.3 follows students who enrolled at any point. 
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Table 3.3 


Course Enrollment and Passing Behavior of Houston Community 
College (HCC) Students Through Four Semesters, by ABE Level, 
Conditional on Enrolling in At Least One Semester 


Course Enrollment and Passing Behavior ABE Level 4 (1) | ABE Level 5 (2) 
Total number of ABE test-takers 1,425 1,979 
Ever took Bridge (%) 50 9 
Passed Bridge 32 7 
Ever took Foundations (%) 40 63 
Passed Foundations 21 33 
Ever took other developmental math (%) 16 29 
Ever passed other developmental math 11 19 
Ever took other developmental math, among those who 65 67 


passed Foundations (%) 
Ever passed other developmental math, among those 


who passed Foundations ae a5 
Ever took college-level math (%) 9 16 
Passed college-level math 6 11 


SOURCE: Calculations from transcript and placement data from Houston Community College and 
placement data from the College Board. Derived from data provided by the College Board. Copy- 
right © 2013-2019 The College Board (www.collegeboard.orq). 


NOTES: Cohorts 1 and 2 are pooled. Sample is restricted to students who enroll at HCC for at 
least one semester and is tracked through four semesters (a larger sample than columns 3 and 4 
of Table 3.2). For Cohort 1 they are tracked through the summer of 2017 and for Cohort 2 through 
the fall of 2017. The pass rate for an individual course is the percentage passing over the percent- 
age who ever took the course. For example, the pass rate for Foundations among ABE Level 4 
test-takers is 52.5 percent (0.21/0.40). Passing Bridge Math is the equivalent of reaching ninth- 
grade math skills. The “among those who passed Foundations” rows are additionally conditional 
on a student passing the Foundations course. For ABE Level 4, the sample size is 294, and for 
ABE Level 5, the sample size is 661. 


by the end of four semesters and 16 percent had enrolled in a college-level math course. Thirty- 
three percent of Level 5 students had passed Foundations of Math within four semesters compared 
with 21 percent of Level 4 students. With 11 percent passing college-level math, there was a 5 
percentage point difference in the rate of Level 5 students compared with Level 4 students passing 
college-level math by the end of four semesters. In sum, very few students who first scored at 
either Level 4 or Level 5 were able to progress through even the first level of college math within 
the two-year follow-up period. 


The Effect of Course Referral on Progress To and Through 
Developmental Math 


In addition to the descriptive data presented above, this study endeavored to estimate the 
effect of being referred to the Bridge course versus being referred to the Foundations of Math 
course on subsequent student outcomes for students with similar math abilities. The goal was 
to use a regression discontinuity design to identify and estimate that effect. Under certain 
conditions, an RDD estimator is unbiased. After working with the data and understanding the 
implementation of the program, however, the research team determined that conditions 


23 


ultimately could not be met for the proposed analysis and, as a result, decided not to include 
the RDD analysis in this report.” 


"2A technical supplement detailing the attempted RDD analyses is available upon request by sending an e- 
mail to angela.boatman@vanderbilt.edu. 
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Chapter 4 


Conclusion and Policy Implications 


Community colleges across the country are experimenting with new approaches to serving stu- 
dents who enter college with math skills so low that they cannot meet the prerequisites for many 
college-level courses, including college-level math, as many students are not well served by tra- 
ditional placement methods and developmental course sequences.'! Many students fail to enroll 
in such developmental courses, and, when they do, fail to pass them — especially those with skills 
below ninth-grade proficiency levels. As a result, colleges are increasingly implementing alter- 
natives to this model, including shortening the sequence, providing extra support to students, and 
allowing students to co-enroll in developmental courses and college-level courses. 


This study examined a model implemented at Houston Community College (HCC) that 
required students to enroll in a four-week Bridge course to raise their math skills to the ninth- 
grade level before they enrolled in a full developmental math course. Starting in fall 2015, the 
Bridge course was offered to students whose math test scores ranged from the fourth-grade level 
to the eighth-grade level. Students who completed the four-week Bridge course had the oppor- 
tunity to immediately enroll into the first course of the developmental math sequence to work 
toward raising their skills to twelfth-grade proficiency. 


A descriptive analysis of student records for two cohorts of HCC test-takers in the 2015- 
2016 academic year yielded several findings about the behavior and outcomes of students who 
tested near the ninth-grade cut point (at Adult Basic Education [ABE] Diagnostic Levels 4 and 
5), including enrollment in college, enrollment in math courses, and progress to and through de- 
velopmental math. 


1. The new placement policy, which targeted students who tested below the ninth- 
grade level, affected one-fifth of students in the sample of placement test-takers. 
Of the approximately 14,000 students who took the math assessment test, almost 40 
percent scored substantially below twelfth-grade proficiency and were automatically 
administered the ABE Diagnostic to determine their math course placement. The ma- 
jority of these (nearly 90 percent) tested at Level 4 and Level 5, corresponding to 
below and above ninth-grade proficiency, respectively. Level 4 students (along with 
the small percentage assigned to Level 3) were referred to the Bridge course — a 
change from the course they would have been referred to under the former policy. 
Level 5 students (along with the handful of students assigned to Level 6) were re- 
ferred to Foundations of Math, the first full course in the developmental math se- 
quence. The few students whose proficiency was assessed below fourth grade (Lev- 
els 1 and 2) were referred to workforce or other programs. In total, one-fifth of all 
Texas Success Initiative (TSI) test-takers were affected by the new assessment and 


| Adelman (1999). 
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placement policies by receiving a score (4 or below) that placed them below the cut- 
off point for developmental math. 


Many students did not enroll at HCC after taking the test. One semester into the 
study period, 53 percent of students who first tested at Level 4 and 48 percent of 
students who first tested at Level 5 had not enrolled in HCC. By the end of four 
semesters, 35 percent of Level 4 and 29 percent of Level 5 test-takers had still not 
enrolled at HCC. Although some of these test-takers were high schoolers who did 
not intend to enroll, some of this drop-off also likely represents students who were 
interested in HCC (or college more generally) but still failed to enroll. 


Even among students who enrolled at HCC, many delayed taking math. More 
than half (55 percent) of enrolled Level 4 students and 41 percent of enrolled Level 
5 students did not enroll in a math course during their first semester. This finding is 
consistent with other research that has shown that developmental students tend to put 
off enrolling in math. 


Many students did not enroll in their assigned math course. Although test-takers 
with scores at Level 4 were somewhat more likely to enroll in Bridge Math and less 
likely to enroll directly into Foundations of Math in the first semester, compliance 
with course referrals was generally low across both groups. After four semesters, 
many students still had not enrolled in the math class associated with their ABE score. 
By the end of four semesters, 51 percent of enrolled students who first tested at ABE 
Level 4 had enrolled in the Bridge course, while 68 percent of students who first 
tested at Level 5 had enrolled in the Foundations of Math course. A limited portion 
of this noncompliance may be due to students retaking the assessment test and being 
placed into a higher-level class; however, this applies to, at most, 11 percent of the 
sample who retested and received a higher score. Findings from the field research 
suggest limited awareness among advisers and some instructors about the depart- 
ment’s intentions to have successful Bridge completers enroll in Foundations con- 
secutively and within the same semester — which likely played a role in most stu- 
dents not knowing to do so. 


Many students did not complete the developmental course sequence. After four 
semesters, a total of 21 percent of all Level 4 students had passed Foundations of 
Math and 6 percent had passed a college-level math course. Students originally test- 
ing at Level 5 fared only slightly better, with 33 percent passing Foundations and 11 
percent passing a college-level math course. 


Policy Implications 


The descriptive results summarized above may help inform community colleges’ decisions about 
options for serving students who enter college with very low math skills. The data and evidence 
presented in this report suggest that, although some students enrolled in the four-week Bridge 
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Course gained the math skills to prepare for the Foundations course, the majority of those students 
referred to the Bridge course did not enroll in the course, and those who did enroll did not neces- 
sarily move on to take or pass Foundations. The study reveals the difficulty of helping these low- 
scoring students. What do these findings mean for postsecondary policymakers and practitioners 
as they weigh their options for serving college applicants with math skills at the primary and 
middle-school levels? 


Several implications for the field from this study are worth noting. First, many students 
who score below ninth-grade proficiency fail to enroll in college and, even if they do enroll, fail 
to enroll in their assigned math course. If the goal is to help students gain math literacy and move 
on to college-level courses, the first step — not taken by a surprising number of students — is to 
get them to enroll in math courses. Colleges could consider revising the “messaging” around the 
importance of taking the math course through traditional advising and orientation sessions, as 
well as through information provided on their websites and social media outlets. Colleges might 
also consider requiring students to enroll in developmental classes immediately upon entry or 
establish a default scheduling approach so that students have a narrower, yet clearer, set of courses 
that direct them through their intended majors or fields of study. 


College decisions about how to serve students with low math scores can be informed by 
further research on the academic behaviors and outcomes of this group of students. The current 
study is purely descriptive; institutional researchers can conduct similar analyses of common 
characteristics tied to students who score low in math placement tests, common “stop out” points 
during their enrollment process, and the distribution of scores within and between the math place- 
ment cut scores. Additional analyses can inform decisions about how to structure placement 
guidelines, the number of seats needed to accommodate different groups of students, and instruc- 
tor deployment. 


Third, colleges already know that new programs and courses, especially those that devi- 
ate from the normal or typical offerings, often require more time than expected to fully implement. 
This study documents that familiar story. As found in the field research, students, instructors, and 
advisers were not initially uniformly aware of the new placement policies and program changes. 
This contributed to lower-than-expected enrollment rates in the Bridge course and still lower tran- 
sition rates to the developmental course for those who passed the Bridge course. Clear vertical 
and lateral communication among the administration, advisers, instructors, and students might 
have alleviated this situation. In particular, access to high-quality advising that consistently com- 
municates course requirements, sequencing, and options is essential for promoting student suc- 
cess generally and is particularly critical when a major change in placement rules such as this one 
is implemented, affecting so many incoming students’ chances at making it to and through devel- 
opmental math and going on to earn their college credentials. 
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Appendix A 


College and Course Enrollment Behavior of 
ABE Test-Takers After One Semester, by Cohort, 
Houston Community College 


Appendix Table A.1 


College and Course Enrollment Behavior of ABE Test-Takers 
After One Semester, by Cohort, Houston Community College (HCC) 


Cohort 1 Cohort 2 
(Expected Start (Expected Start 
Fall 2015) Spring 2016) 
Behavior Level 4(1) Level 5 (2) Level 4 (3) Level 5 (4) 
Number of ABE test-takers 1,490 1,926 714 846 
Behavior at end of first semester after ABE 
assessment (%) 
Did not enroll at HCC 55 52 47 39 
Enrolled at HCC, did not take any math 25 20 28 26 
Enrolled, took Bridge only 10 2 16 2 
Enrolled, took Foundations only 6 24 3 28 
Enrolled, took both Bridge and Founda- 
tions 4 1 5 3 
Enrolled, took other developmental math 0 1 1 1 
Enrolled, took college-level math 0 1 0 1 


SOURCE: Calculations from transcript and placement data from Houston Community College and placement data from the 
College Board. Derived from data provided by the College Board. Copyright © 2013-2019 The College Board 


(www.collegeboard.org). 


NOTES: Cohorts 1 and 2 are pooled. A student's first semester after taking the ABE assessment is the fall of 2015 for 
Cohort 1 and the spring of 2016 for Cohort 2. For Cohort 1 the end of the second semester is the summer of 2016. 
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